Attention for Fine-Grained Categorization

This paper presents experiments extending the work of Ba et al. (2014) on recurrent neural models for attention into less constrained visual environments, beginning with fine-grained categorization on the Stanford Dogs data set. In this work we use an RNN of the same structure but substitute a more powerful visual network and perform large-scale pre-training of the visual network outside of the attention RNN. Most work in attention models to date focuses on tasks with toy or more constrained visual environments. We present competitive results for finegrained categorization. More importantly, we show that our model learns to direct high resolution attention to the most discriminative regions without any spatial supervision such as bounding boxes. Given a small input window, it is hence able to discriminate fine-grained dog breeds with cheap glances at faces and fur patterns, while avoiding expensive and distracting processing of entire images. In addition to allowing high resolution processing with a fixed budget and naturally handling static or sequential inputs, this approach has the major advantage of being trained end-to-end, unlike most current approaches which are heavily engineered.